International Journal of ePortfolio 
http ://www. theij ep. com 


2014, Volume 4, Number 2, 143-156 
ISSN 2157-622X 


A University-Wide ePortfolio Initiative at Federation University Australia: 
Software Analysis, Test-to-Production, and Evaluation Phases 

Rachael Hains-Wesson Lara Wakeling and Peter Aldred 

Deakin University Federation University Australia 

This paper describes an ePortfolio implementation strategy at Federation University Australia, 

Victoria (formerly the University of Ballarat). The authors combined a personal and practical 
viewpoint to elicit pitfalls, challenges, and recommendations for improvement. The paper is divided 
into three main areas in order to outline the experiments that occurred. The first section provides a 
standard literature review around ePortfolio adoption as well as a research-based analysis of 
available ePortfolio software at Australian universities. The second part depicts the University's 
ePortfolio implementation strategy that focused on “test-to-production” and technology 
dissemination phases. This section is based on the authors' personal viewpoint of ePortfolio adoption 
at a university where a “top-down management decision making model’’ (Slade, Murfin, & 

Readman, 2013, p. 178) was used. Third, the evaluation strategy is reported, which was based on 
similar research conducted at Australian universities (Hallam & Creagh, 2010; Hallam, Harper, 

Hauville, Creagh, & McAllister, 2009). This part is offered as a modest-scoped, mixed methods 
evaluation process. The paper extends on ePortfolio implementation strategies (Bell & White, 2013; 

Coffey & Ashford-Rowe, 2014; Jarrott & Gambrel, 2011; Lambert & Corrin, 2007; Ring & 

Ramirez, 2012; Slade et ah, 2013) and software analysis (ACODE, 2011; Slade et ah, 2013). 
Recommendations are made for the careful integration of pre- and post-rollout of ePortfolio 
programs with face-to-face ePortfolio tutor support, offering online resources and alternative 
portfolio-making options for students with poor broadband access. 


Literature Review 

The practice of ePortfolios in higher education and 
in the vocational education and training sector has 
increased steadily over the years (Coffey & Ashford- 
Rowe, 2014; Galatis, Leeson, Mason, Miller, & 
O’Neill, 2009; Ring & Ramirez, 2012). However, 
research indicates that the term ePortfolios “will not 
encompass every possible permutation” (Galatis et ah, 
2009, p. 6; Hallam & Creagh, 2010), its uses (Hallam & 
Creagh, 2010; Lambert & Corrin, 2007) or its purposes 
(Whilhelm et ah, 2006). It would appear that some 
research around ePortfolio pedagogy and technology 
describes the general characteristics and different types 
of ePortfolios, users’ experiences, attitudes, and 
opinions alongside providing opportunities for learners 
to demonstrate evidence of individual achievements, 
such as employability skills and graduate attributes 
(Barrett, 2007; Hallam & Creagh, 2010; Stefani, 
Mason, & Pegler, 2007; Strivens, 2007; Wade, Abrami, 
& Sclater, 2005). Other research provides numerous 
examples of the benefits of ePortfolios for fostering 
students’ sense of pride in their work (e.g., Sherry & 
Bartlett, 2005), increasing students’ learning motivation 
(e.g., Tosh, Penny Light, Fleming, & Haywood, 2005), 
and viewing ePortfolios as a way for students to share 
their work with others (e.g., Wilhelm et ah, 2006). 

Given the many proven learning and teaching 
benefits of ePortfolios in higher education, there is a 
need also to understand that to ensure success with 
implementing ePortfolios as a long-term project, time is 
required to perform an initial study on the experience, 


challenges, and issues, after which wider 
implementation can occur. However, effective 
technological adoption and undertaking of a wide 
evaluation process can be problematic for those who are 
at the “coal face” of ePortfolio adoption when a “top- 
down management decision making model” (Slade et 
ah, 2013, p. 178) is used, or when time and financial 
constraints are of importance. Concomitantly, a review 
of previous university-wide ePortfolio initiatives is 
therefore beneficial (Coffey & Ashford-Rowe, 2014; 
Hallam & Creagh, 2010; Hallam et ah, 2009; Lambert 
& Corrin, 2007; Slade et ah, 2013). 

The study reported in this paper was influenced by 
research advocating the importance of reviewing 
learning design and existing technologies in higher 
education, particularly in the area of enterprise (e.g., 
Ferdig, 2005; McAfee, 2006; Salmon, 2011). 
Notwithstanding, it was also motivated by the literature 
that showcases an understanding of technology 
implementation strategies, such as considering the 
existing culture of an organization (e.g., Conole, White, 
& Oliver, 2007) in the midst of significant change (e.g., 
Ferdig, 2005; Henriksen, 2002) and the necessity of 
“alignment to institutional and user needs” (Slade et ah, 
2013, p. 177). 

ACODE (2011), Butler (2006), Himpsl and 
Baumgartner (2009), and Uys (2007) have extensively 
reviewed the success criteria for an ePortfolio system 
and identified key elements. In addition, alternatives to 
traditional ePortfolio platforms such as websites, blogs, 
and wikis have also been recognized, but with different 
implications for learners and institutions (e.g., 
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Electronic Portfolio Action and Communication Wiki 
Space, 2013). Moreover, based on the work conducted 
in the UK as part of the Joint Information Systems 
Committee (JISC, 2008) ePortfolio investigation, a 
series of five threshold concepts were suggested to 
ensure successful institutional adoption of ePortfolios. 
These were: the purpose, learning activities design, 
process, ownership, and disruptive nature of ePortfolios 
(JISC, 2008; Joyes, Gray, & Hartnell-Young, 2010). 
Furthermore, the success criteria for ePortfolio 
implementation relating to students (curricular) has 
been outlined by Butler (2006), the Australian 
ePortfolio Project (Hallam et al., 2009), and Joyes et al. 
(2010), who proposed that ePortfolio-making is more 
likely to occur with support from the university’s 
highest level, such as learning and teaching committees, 
internet technology staff, and faculty leaders (Allan & 
Cleland, 2012). Alternatively, Slade et al. (2013) 
advocated a middle-out approach rather than a top- 
down approach to technology adoption. By contrast, 
within the Faculty of Education at the University of 
Tasmania, Australia, the creation of a community of 
practice for the learning management system was 
initially established, in which students could have all 
questions related to ePortfolios answered from both a 
technical and educational perspective (Allan & Cleland, 
2012). The community of practice, along with a 
structured workshop series, scaffolding of activities, 
and the provision of templates were key to the 
successful implementation three years on. 

The study noted in this paper focuses on the 
approach reported by Parker (2010), in which a lecture 
style was used for training, along with the option of 
seeking additional help, and was found to be 
inadequate. Instead, students viewed demonstration and 
training in a computer laboratory as more 
advantageous. This research project takes as its lead 
recent studies that advocate such notions and also 
identify key barriers to ePortfolio technological 
implementation (Butler, 2006; Owen, 2009). These are: 

• The need for and access to adequate hardware 
and software that is maintained; 

• An awareness of the lack of technological 
skills amongst staff and students; 

• Addressing technical problems and support for 
staff and students; 

• Adequate storage space and server reliability; 

• Demands on staff time; 

• Efficient use of student time; 

• Ownership and technological issues; 

• Security and privacy of data; 

• Appropriate features and control over them; 

• Access and permission controls; 


• On-going access for students upon completion 
of their course. 

It would appear that the substantial literature and 
reports on ePortfolio practice and pedagogy illustrate a 
breath of implementation overview, practice, users’ 
experience, and users’ advice (e.g., Chau & Cheng, 
2010; Dinmore, Kherwald, & Bradford, 2011; Halstead 
& Wheeler, 2009; McNeill & Cram, 2011; Taylor, 
Dunbar-Hall, & Rowley, 2012; Whilhelm et al., 2006). 
However, there is still immense scope for further 
research into specific implementation methods that 
expand on the literature. More specifically, in Australia 
there is limited research available on the practice of 
implementing ePortfolios at a university-wide level 
where a “top-down management decision making 
model is used” (Slade et al., 2013, p. 178; e.g., Coffey 
& Ashford-Rowe, 2014; Hallam & Creagh, 2010; 
Hallam et al., 2009; Lambert & Corrin, 2007). 
Therefore, this study addresses the critical issue of 
selecting an institution-wide ePortfolio system based on 
the following: (1) a top-down management decision 
making model; (2) significant time and financial 
constraints; (3) limited technological support; and (4) 
the requirement to implement a centralized ePortfolio 
system for learners. 

Federation University Australia is a rural, dual¬ 
sector university and the third oldest site of higher 
learning in Australia, with approximately 23,500 
domestic and international students enrolled each year. 
Many of these students are the first in their family to 
attend university. The institution offers secondary 
schooling, vocational education and training, higher 
education, and post-graduate studies. Thus, the 
ePortfolio software must be appropriate for a wide 
range of disciplines, educational contexts, accreditation 
needs, and have the ability to showcase a wide range of 
artifacts and evidence of learning, as the software will 
be used by students from both the vocational education 
and training and higher education sectors. With this in 
mind, it is important to consider that rural universities 
do not tend to have the resources (and/or finances) 
readily available for such undertakings compared to the 
larger capital city-based institutions. Therefore, in this 
study, the success or otherwise of ePortfolio 
implementation is dependent on the perspective of 
those involved. As Joyes et al. (2010) and Voigt (2009) 
noted, while technical support may be important to 
some, it will be pedagogical support that is more 
important to others. Finally, the study reported in this 
paper contributes to ePortfolio implementation 
strategies, innovative technologies in education, and 
inherent challenges (e.g., Bell & White, 2013; Burnett, 
2001; Jarrott & Gambrel, 2011; Lambert & Corrin, 
2007; Ring & Ramirez, 2012; Slade et al., 2013). 
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Limitations 

One of the major limitations of this study is that 
the technological innovation adoption (test-to- 
production) was made over a limited period of time. 
This was mainly due to staff turnover, particularly in 
the area of technical ePortfolio support as well as 
institutional time and financial constraints (due to 
external factors). A more stringent test of adoption of 
an innovation, or success of an implementation plan, 
would be needed to ascertain its continuation or 
persistence over a longer period, such as several years. 
The results presented here do not show data collected 
over a longer period of time. However, because of the 
importance of exploring innovation adoptions over an 
extended period of time, a further mixed methods 
research ePortfolio project is currently being 
completed at the university. This, in turn, will assist in 
furthering conversations, evaluation outcomes, and 
recommendations, as well as the promotion of 
ePortfolio adoption at the university. 

Implementation 

In the following sections, the method and results of 
the implementation phases of the ePortfolio adoption 
are outlined, noting the various successes and 
challenges. This is achieved by describing the 
information under three main headings: (1) ePortfolio 
software at Australian universities, where information 
about the choice of software is presented; (2) 
implementation strategy; and (3) a modest-scoped 
evaluation process, which was based on similar 
research conducted at Australian universities (e.g., 
Hallam & Creagh, 2010; Hallam et al., 2009). 

ePortfolio Software at Australian Universities 

Choosing the right ePortfolio platform is no easy 
matter. For instance, deciding on an appropriate 
university-wide ePortfolio platform can vary 
extensively from institution to institution (Conole et ah, 
2007; Goldsmith, 2007; Slade et ah, 2013). For 
Federation University Australia, implementing a 
university-wide tool proved challenging, with the 
following points of particular concern: 

• How will ePortfolios be used? 

• Who will use ePortfolios? 

• How to avoid smothering innovation and 
creativity (Slade et ah, 2013); 

• Who is the intended audience? 

• How to ensure that the software meets the 
needs of all users; 


• Ascertaining the costs and availability of 
hardware and software resources (ACODE, 
2011; Butler, 2006; Himpsl & Baumgartner, 
2009; Uys, 2007). 

Conversely, Himpsl and Baumgartner (2009) stated 
that in 2008 there were over 60 ePortfolio providers to 
select from, which can make decisions even harder. For 
instance, Slade et ah (2013) highlighted the difficulties 
of deciding on an ePortfolio platform due to the 
extensive list of ePortfolio solutions that are now 
available, and “chos[ing] to only consider options for 
which there was either experience available with the 
university and more generally available in the higher 
education sector” (p. 180). Himpsl and Baumgartner’s 
(2009) ePortfolio software categories and the 
Educational Technology Survey report (ACODE, 
2011), which was conducted by Sarah Lambert, were 
important factors for the University of Southern 
Queensland’s ePortfolio decision-making strategy 
(Slade et ah, 2013). Similarly, the abundance of 
alternative choices to traditional ePortfolio platforms 
that are presented on the Electronic Portfolio Action 
and Communication Wiki Space (2013; e.g., 

Wordpress, Google Apps, and wikis) further creates 
various issues around ePortfolio decision-making. 

For this research project, it was the ACODE (2011) 
and Himpsl and Baumgartner (2009) reports that were 
used, aiding conversations and recommendations 
regarding the University’s senior management culture 
for decision making. The reports exhibited the group of 
leading ePortfolio software in the higher education area, 
namely Mahara and Pebblepad, which presently are the 
most frequently used (traditional) ePortfolio platforms 
in Australia’s higher education sector. However, the 
reports did not emphasize each university’s ePortfolio 
software in detail. Due to the ever-evolving 
technological ePortfolio landscape in Australia’s higher 
education, the present authors conducted a World Wide 
Web internet search of 35 Australian universities’ 
homepages to gain an “Australian-centric” ePortfolio 
users’ perspective (see Appendix). The internet search 
review was deemed advantageous by the University’s 
senior management so that technology adoption 
software that was currently being used in Australia 
could be analyzed, with the information gained being 
used for official decision-making. The review process 
was based on a similar collection method conducted by 
Hains-Wesson (2012), in which “Australia’s and the 
United Kingdom’s universities’ homepages were 
located on the World Wide Web ... in order to present 
a detailed account of online creative journals that 
operated as Work-Integrated Learning activities” (p. 
267). The method was adapted to fit the research 
project’s purpose. The review was completed by first 
utilizing the list of Australian universities presented on 
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the ACODE (2011) survey report and then manually 
completed an internet search of each university’s 
homepage with the key search words “ePortfolios,” 
“portfolios,” and/or “education portfolios.” The process 
enabled the researchers to locate current ePortfolio 
information. To account for any missing information, a 
member from the research team telephoned each 
university’s Learning and Teaching Centre and/or sent 
an e-mail to the appropriate manager to request and 
obtain the data. The data presented in the Appendix 
corresponds to the ACODE (2011) survey results, but 
with the addition of specific ePortfolio platform details 
from each university. The authors extracted specific 
data relating to rural, dual sector universities from the 
information presented in the Appendix to help address 
the broader needs of students at Federation University 
Australia (see Table 1). 

At the time, a close analysis of the literature around 
ePortfolio software analysis (ACODE, 2011; Himpsl & 
Baumgartner, 2009), alongside the data presented in 
Table 1, suggested an even spread of Mahara, 
Pebblepad, and other ePortfolio use at Australian 
universities. Using the data from the reports and the 
internet search review process, the information was 
presented to the University’s senior management team, 
providing extra evidence. In addition, senior 
management requested the following additional criteria 
to be considered: 

• The ability of the software to deliver a diverse 
range of institutional reporting needs, such as 
demonstrating competency and accreditation 
requirements and showcasing a variety of 
artifacts; 

• The ability of the software to be integrated 
with the university’s learning management 
system (Moodle); 

• That the software should be open source and 
have an active community of practice; 

• The ready availability of introductory online 
resources (e.g., information on how to set up 
an ePortfolio); 

• The ability of the software to provide built-in 
tools such as content management and plug-ins 
for mobile use within the university’s learning 
management system (Moodle); 

• A history of other universities active use of the 
ePortfolio software in order to establish a 
community of inquiry across institutions, 
encouraging knowledge-sharing. 

Moreover, when implementing new technology, the 
university’s context, management culture and goals are 
key when making decisions. Evans and Benefield 


(2001) pointed out that to evaluate ePortfolio adoption 
properly, it is beneficial to undergo an extensive review 
in order to make good decisions. With this in mind (and 
the above review method and points taken into 
consideration), Mahara was chosen to be the official 
ePortfolio platform. 

First, it met the university’s criteria. Second, it was 
an educational and functionally effective solution for 
Federation University Australia because of its open 
source software licence, ease of compatibility into the 
university’s LMS (Moodle), and focus on being a 
personal learning environment that mixes with social 
networking and allows users to easily collect, reflect on, 
and share their achievements. However, the university’s 
decision to purchase an open source ePortfolio software 
such as Mahara, with its active community of practice 
and history of other universities using the software, did 
not mean fewer challenges. These challenges are 
detailed in the following section in terms of the 
implementation strategy employed. 

The Implementation Strategy 

As mentioned previously, the following 
implementation strategy was based on a top-down 
management decision model rather than a “middle 
agent feasibility study” (Slade et al., 2013, p. 178). It is 
also important to note that in early 2012, when senior 
management decided that a Mahara system was to be 
implemented as the university-wide ePortfolio system, 
a test phase was established and opened up to interested 
teachers and students to experiment with on an ad hoc 
basis. The majority of ePortfolio users in the test 
environment were from the disciplines of education and 
nursing. This outcome aligns with research around 
specific disciplines that are most likely to take-up 
ePortfolio adoption quickly (Bashook, Gelula, Joshi, & 
Sandlow, 2008; Maher & Gerbic, 2009) compared to 
others areas of study, such as engineering or 
mathematics (Carroll, Markauskaite, & Calvo, 2007). 
By late-2012, and after much deliberation with 
numerous university stakeholders, the university’s 
Learning and Teaching Committee released a document 
titled “Learning and Teaching Plan 2012-2014” 
(University of Ballarat, 2012). In this document, the 
authors noted that ePortfolios were one of the key 
eLearning tools to benefit learning for reflective 
practice, assessment, and evidence learning (Slade et 
al., 2013). University of Ballarat (2012) also presented 
key performance indicators for the university-wide 
ePortfolio initiative, anticipating that students’ active 
use of ePortfolios would increase by the end of 2015. In 
order to meet the University of Ballarat’s (2012) 
objectives, meet the Federation University Australia’s 
strategic learning and teaching outcomes, and to go 
effectively from test-to-production, it became essential 
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Table 1 


Summary of ePortfolio Software Platforms Supported in Australia’s Rural 
_ and Dual Higher Education Sector in 2014 _ 


Name of institution 

Mahara 

Pebblepad 

Other 

Central Queensland University*** 

V 

X 

X 

Charles Darwin University** 

X 

X 

Web2.0 

Royal Melbourne Institute of Technology* 

X 

V 

X 

Southern Cross University*** 

X 

V 

X 

University of Sunshine Coast*** 

X 

V 

X 

The University of New England*** 

V 

X 

X 

Victoria University* 

X 

X 

Desire2Leam 


Total 2 

3 

2 


Note. V = Used. X = Not used. “Swinburne University of Technology are two of only five dual-sector institutions in Australia, of which 
**Charles Darwin University is the only one outside Victoria. The others are *Royal Melbourne Institute of Technology and *Victoria 
University. ***There are five rural universities in Australia. 


that a dedicated staff member be made responsible for 
the rollout. The requirement to support test-to- 
production with pedagogical and internet technology 
support was enacted, and providing support structures 
for teachers and users was highly recommended. As a 
consequence, a working party was created to encourage 
innovation and individual staff members selected to 
introduce change and encourage peers to follow 
(Cummings, Phillips, Tilbrook, & Lowe, 2005). The 
working party included key ePortfolio stakeholders, 
such as those who had influenced the test phase, and 
staff members such as internet communications 
technology staff, administrators, academics, general 
staff, and one student representative with the 
knowledge that ePortfolios can “enhance teaching, 
learning and assessment practices” (Davis & Murrell, 
1994, p. 2). The formation of the working party also 
aligned with the recommendations made by the 
Australian ePortfolio Project (Hallam et al., 2009), as 
well as the successful university-wide ePortfolio project 
that had occurred at the University of Wollongong, 
New South Wales, Australia (Lambert & Corrin, 2007). 
The members of the working party were also 
responsible for communicating recommendations to 
senior management, the test-to-production phase, a 
modest-scoped evaluation, and the following specific 
outcomes: 

• Review, evaluate, and fix any errors associated 
with the Mahara test environment, ePortfolio 
software, and hosting system; 

• Plan, develop, and implement an appropriate 
and effective production rollout framework 
(i.e., test-to-production); 

• Organize the website branding of the open 
source software to coincide with the 
university’s webpage design and policy; 


• Develop best practice internet technology 
upgrades, solutions, and website hosting 
procedures post-test-to-production phase; 

• Increase the membership of the working party 
to enable more expert advice on the effective 
use of ePortfolios and implementation across 
the rural dual-sector university; 

• Create online resources that are user friendly 
and relevant; 

• Communicate, plan, and implement ePortfolio 
workshops and seminars around internet 
technology and pedagogy for students and 
teachers, particularly those who were new to 
ePortfolios. 

In reality, however, the above objectives were not 
free from hindrance. For instance, the test-to-production 
phase took place at a time when the Victorian State and 
the Federal Governments had announced financial 
cutbacks that affected the vocational education and 
training and the higher education sectors in Victoria, 
Australia. The Victorian Government (the Baillieu 
Government at the time) “had slashed $290 million 
from the sector, with around the same amount to be cut 
in both 2013 and 2014” (Rea, 2012, p. 16). It was also 
essential that the rollout plan occurred in order to meet 
the University of Ballarat’s (2012) learning plan 
outcomes, the requirements of the Australian Quality 
Framework (the national policy for regulated 
qualifications in Australian education and training), and 
the Tertiary Education Quality and Standards Agency 
(Australia’s independent national regulator for the 
higher education sector) by 2015 (Milne, Heinrich, & 
Lys, 2010). In order to provide a high quality ePortfolio 
experience for students and staff that met the above 
points at a financially difficult time, the formation and 
involvement of the working party was imperative. In 
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addition, the ePortfolio platform required an 
appropriate upgrade, as the test environment had 
already alerted the university’s technological support 
staff to various errors in the program that were 
occurring at the time, such as a failure to embed certain 
files and videos successfully, and sporadic text 
disruptions throughout users’ ePortfolios. Website 
branding also needed to occur and the online resources 
and policies associated with privacy, assessment, and 
security needed to be updated. In other words, it was 
important to get the technology right before ePortfolios 
could be successfully rolled out on a university-wide 
level. One of the main challenges during this time was 
keeping in mind the established users within the test 
system. Without internet technology being stable and 
up-to-date, there were risks of ePortfolio test- 
environment users becoming ePortfolio-fatigued and/or 
lacking ePortfolio creation motivation. Additionally, 
without effective internet technology support, it would 
be difficult to meet the university’s curriculum-renewal 
strategy for undergraduate degrees (University of 
Ballarat, 2011). As one of the key recommendations 
stated, 

The University will ensure that all students are 
provided with the opportunity to record the 
development of their graduate attributes and that 
attainment of these be assessed, where possible and 
applicable, as part of the curriculum using an 
assessable portfolio which is part of a final course. 
(University of Ballarat, 2011, p. 1) 

With the decision to rollout Mahara as the official 
ePortfolio software for Federation University Australia, 
and to therefore go from test-to-production, it became 
necessary to offer workshops and seminars on 
ePortfolio making for new users on assessment practice 
and on showcasing and evidencing learning. This 
needed to occur within a constricted time frame, which 
was approximately eleven weeks. Ultimately, the 
working party needed to keep abreast of the 
University’s assessment due dates across the teaching 
landscape, so as to minimize any disruption due to the 
upgrade from test-to-production, notwithstanding the 
fact that successful broader implementation of 
ePortfolios can be dependent on evaluation. 

In the following sections, the modest-sized 
evaluation phase of the project is presented, which was 
based on the authors’ viewpoints from the participants’ 
experiences in workshops and seminars. 

The Evaluation Process 

Participants who were new to ePortfolio-making 
were encouraged to take part in the evaluation phase of 
the project. Users within the test environment were also 


invited to participate. This was achieved through 
notification of the opportunity to participate in 
ePortfolio workshops and seminars via posters, word- 
of-mouth recommendations, networks, and program 
presentations. The ePortfolio initiative was modestly 
evaluated due to the majority of participants being self- 
selected and new to ePortfolios. The workshops centered 
on information about specific challenges for new users to 
the system and how to use ePortfolios for an assessment 
task such as reflective practice for teachers, course 
accreditation requirements, and the showcasing of 
artifacts. The majority of participants (n = 34) had no 
prior knowledge of ePortfolios (96%). They were keen to 
find out what an ePortfolio could offer in terms of 
professional development, such as research, networking, 
and/or the evidencing of achievements as part of the 
student learning experience. Student participants (n =18) 
were invited by their teacher/s to take part in the 
ePortfolio workshops because reflective practice via 
ePortfolio was being introduced as a major assessment 
item. Similar to those for staff, student ePortfolio 
workshops were often presented in computer 
laboratories, with three additional seminars being 
presented in a classroom. On request, all workshops and 
seminars for staff and students were repeated when 
requested within the 11-week period. 

Method 

The Australian ePortfolio Project’s (AeP, 2010) 
learning and teaching survey was modified for the 
project’s modest evaluation phase. Minor changes were 
made to the AeP survey, with the following research 
questions guiding these changes: 

• What factors do you believe will help you to 
implement/develop an ePortfolio? 

• What factors do you believe prove or are 
proving difficult for you to implement/develop 
an ePortfolio? 

• If you have developed an ePortfolio, what 
impacts do you believe have occurred as a 
result of your ePortfolio use? 

Participants 

The participants (n = 52) consisted of 89% from 
the higher education sector, with the remaining from 
the vocational education and training area. There was 
an even mix of male and female participants. 

Survey 

The construction of the survey was based on 
similar research conducted at Australian universities 
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(Hallam & Creagh, 2010; Hallam et al., 2009). The 
survey consisted of a mixture of eight open-ended and 
closed questions. 

Workshop Implementation 

It is important to note that each workshop was 
purposely delivered according to the needs of the 
participants. For example, some presentations were 
conducted in a seminar room, where the presenter 
showed information to the audience via a large 
computer screen, or in computer laboratories where 
participants could actively try out the new information 
being learned. At each presentation, the following main 
topics were focused on: (a) an introduction to the term 
ePortfolio; (b) the benefits of creating and sustaining an 
ePortfolio; (c) where to find online resources for the 
novice ePortfolio designer; and (d) showing examples 
of good practice ePortfolio-making from the active test 
environment already established in the system. 
Additionally, any internet technology concerns that 
participants had were also a priority in the workshops 
(e.g., how users share ePortfolios, how to insert links 
and videos). Each session concluded by inviting 
participants to provide anonymous feedback via the 
survey. The second aim of the evaluation was to gather 
information from new ePortfolio users, such as their 
opinions and experiences. This part of the evaluation 
strategy was important to the research team in order to 
ensure that the technology was functioning at a high 
standard. 

Survey Results 

The participants came from a variety of disciplines 
and areas, such as psychology, sciences, internet 
technologies, business, food science, and the vocational 
education and training sector. There was not enough 
data to disclose gender or age range as a variable. Once 
the feedback from the surveys was collected, the data 
were extracted, de-identified, and placed into an Excel 
spreadsheet. The research team analyzed and used pivot 
tables to provide quantitative analysis, with the 
qualitative responses being manually coded by themes. 
The key themes that emerged from the data were: (a) 
understanding ePortfolio use; (b) issues around 
ePortfolio use; and (c) support around ePortfolio use. 
The following section provides the quantitative analysis 
by theme followed by the qualitative responses. 

Quantitative Results 

Understanding eportfolio use. The ePortfolio 
workshops during the test-to-production phase were 
developed to encourage new and established test 
environment users to feel supported when discussing 


the challenges and benefits of implementing ePortfolios 
in learning and teaching experiences. The workshops 
also provided participants with an avenue to provide 
feedback to the research team in order to instigate 
positive change during the test-to-production phase and 
to communicate recommendations to senior 
management. The workshops were also a good 
opportunity to make sure that the technology was 
functioning at a high standard, as well as to address any 
learning and support gaps. The workshops also assisted 
the researchers in deciding which cohort would be most 
advantageous for a future technological adoption- 
evaluation outcome over a longer period of time. 

One of the main outcomes of the workshops was 
that a number of participants (« = 20) suggested that 
they gained more than a rudimentary understanding of 
ePortfolios, with over 80% of informants noting that 
they felt ePortfolios were very important or important 
to their teaching and learning journey. 

Issues around eportfolio use. Some of the key 
concerns that participants had about implementing 
ePortfolios involved not understanding the software 
program, and they often stated that they were slow to 
learn, especially when internet technological support 
was not present or internet access was slow. Factors 
respondents felt might impede their ePortfolio making 
included: (a) self-motivation and a lack of clear 
direction; (b) a lack of internet access at home; and (c) 
the nature of the Mahara program (e.g., “sometimes I 
find it just confusing, with all the many tabs that 
Mahara has”). 

Support for eportfolio use. The quantitative 
results from the surveys further illustrated that after a 
second or third workshop, users were more aware of 
online assistance being offered rather than the hardcopy 
support resources or face-to-face tutor support. For 
example, 27% versus 54% were aware of online 
resources and tutor support, with over 50% of 
participants being aware of online guidance on how to 
use Mahara, a tutorial program to support the ePortfolio 
process, and the opportunity of face-to-face tutor 
support for learners. Therefore, while less than 30% 
were aware of online tutor/mentor support for feedback 
at the end of a workshop, this number had doubled by 
the time additional sessions were offered to the same 
cohort. For instance, at the completion of a third 
ePortfolio training session participants indicated that 
the factors they believed would help them with their 
ePortfolio assessment were: (a) ease of accessibility, (b) 
confidence with the software program; (c) getting face- 
to-face help from the ePortfolio support staff member; 
and (d) completing the training sessions and receiving 
on-going face-to-face feedback from the lecturer. 

In terms of the workshops, 92% of participants 
found them to be highly effective, and many 
participants (including both teachers and students) 
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noted that they found the workshops worthwhile (e.g., 
“Meeting new people, being able to read other people's 
work for my benefit only—what I compete with 
really—and how I can improve my writing—something 
I’m not strong at”). 

Qualitative Results 

Understanding eportfolio use. A number of 
participants were unaware of what an ePortfolio was or 
its capabilities, saying, for example, that “I had never 
heard of ePortfolios so it was a good introduction and I 
am very interested in starting one” and “I didn’t know 
about ePortfolios until today—will have to explore 
further to know what it might mean for me in the 
future.” The workshops proved useful for new users 
due to the benefit around introducing teachers and 
students to the software as well as its educational 
opportunities that already had an active community of 
practice within the system. For example, by the end of 
the workshop participants were writing insightful 
comments. The following list illustrates participants’ 
comments on the opportunities an ePortfolio could 
offer: “It’s like a blog or social network site where 
students interact with each other”; “[It’s] a place to 
show your professional history”; “[It’s] a tool to 
develop material related to study that may be useful in 
the future”; “[ePortfolio is] an on-line repository of 
personal and professional information that can be 
available to myself and others”; and “Viewing what 
others have already done with their ePortfolios [is 
beneficial] for [developing] ideas.” 

Support around eportfolio use. The overall 
feedback gained from the participants also showed that 
there was a real need for consistent support, such as 
online and face-to-face resources (blended) and 
showcasing (early) good practice (e.g., examples of 
effective ePortfolios in order to fully appreciate its 
value in learning and teaching). The participants’ 
responses suggest that this needed to occur prior to, 
during, and after the test-to-production phase. 
According to JISC (2008), it is necessary to understand 
that the learner’s initial difficulties with both 
technology and reflective learning increase the 
workload of tutors and internet technology support staff 
in the early stages of implementation. This was also a 
key concern that was expressed and observed by the 
research team in the workshops. However, once 
autonomous learning was established, the support 
required from tutors and internet technology staff 
diminished, making more time for proactive 
interventions with individuals around their ePortfolio- 
making endeavors. 

Issues around eportfolio use. One area of concern 
was related to an instructor’s awareness of the 
advantages and disadvantages of ePortfolio assessment. 


On the one hand, digital portfolios offered more 
efficient working practice, enabling marking and 
verification to take place incrementally. Yet, the 
diversity of evidence contained in ePortfolios can make 
them harder and more time consuming to mark. There 
were also concerns raised by participants about 
ePortfolio ownership upon graduation. As a result, the 
feedback from participants assisted the working party to 
solve this particular issue by orchestrating and 
sustaining stronger links with the university’s 
administration process, alumni membership, Careers 
Centre, and Student Experience Officers to grant 
graduates continued use of their ePortfolios after 
graduation. This particular issue and outcome is also in 
line with previous research (Hallam et al., 2009; JISC, 
2008), and ideally, the ePortfolio system should always 
allow users to export their ePortfolio account and/or to 
continue using their ePortfolios as an Alumni member 
to encourage life-long learning. 

Participants also explained that ePortfolio 
construction was only important to them because of a 
course’s requirements of ePortfolio assessment. It was 
often teachers who were also students that made these 
types of comments (e.g., “[Unless] consistent and 
effective feedback is provided, [I would] prefer face-to- 
face contact and an assessment task that could be 
handed in via a hard copy format”). Students regularly 
reflected on being “personally frustrated” with 
ePortfolio making, which was mainly due to inadequate 
feedback and insufficient internet technology support or 
training rather than the ePortfolio platform itself. 

Some participants expressed that using a remote 
and/or poor internet connection influenced their 
decision to avoid completing an ePortfolio assignment. 
In the literature, this is also a common concern, 
particularly in the context of rural universities (e.g., 
Bell & White, 2013). The below quote from a 
participant further reflects this particular point: 

The videos [that help explain how to make an 
ePortfolio] may have been more helpful if they 
would have downloaded faster at home. For some 
reason your videos literally take forever to 
download at our rural property and therefore are 
not accessible for me unless I travelled either to the 
university or my workplace (64 kilometres away). 

Conclusion 

The study described a challenging ePortfolio 
implementation strategy at a rural, dual sector 
University in Victoria, Australia, which was based on 
the researchers’ perspectives via a top-down 
management approach to technology adoption to drive 
change (Cummings et al., 2005). The findings detail an 
ePortfolio implementation process, alongside a modest- 
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scoped, mixed method evaluation, which was impacted 
due to time and financial constraints. Throughout the 
study, the authors combined a personal and practical 
viewpoint to elicit the complex nature and continual 
shifting ground of ePortfolio adoption at an institutional 
level where management culture, financial, 
technological, and pedagogical constraints are of 
importance. According to Rogers (2003), most faculty 
members adopt innovations at individually varying 
rates, and achieving faculty compliance takes time and 
development. For example, when users were asked 
what factors helped them to plan, develop, and sustain 
an ePortfolio in their teaching and learning, the 
majority of participants commented that receiving help 
from peers, perseverance, actively accessing online 
tutorials about ePortfolio use, participating in face-to- 
face workshops, trial and error, a desire to create web 
pages, and previous knowledge of ePortfolios as well as 
technological skills were all highly beneficial. 

Despite the modest number of participants (n = 52) 
who took part in the mixed-method evaluation phase, 
the results have been advantageous to the ongoing 
rollout and improvement of the ePortfolio program at 
Federation University Australia. For instance, the 
following recommendations were developed and were 
based on the participants’ experiences at the workshops 
and seminars: 

• Provide engaging and minimal downloads of 
hypertext links to video or PDF files for all 
online resources, as well as offer face-to-face 
professional development sessions around 
alternative ePortfolio submission practices for 
students with poor broadband access; 

• Provide users with engaging and minimal 
downloads for all online resources to showcase 
alternative ways to evidence learning, such as 
using audio (podcast recordings) that can be 
uploaded when at a campus internet 
accessibility area or via a rural Wi-Fi 
community hub area; 

• Encourage the use of CD-ROM formats for 
critical content that is easily accessible for 
students who do not have effective broadband 
access; 

• Offer students ePortfolio alternative 
submission practices, such as DVD/CD or 
USB modes, especially for those who have 
mobility issues, in order to provide inclusive 
learning practices; 

• Provide ongoing introductory sessions for new 
users, ePortfolio workshops via face-to-face 
sessions around sustaining an ePortfolio, and 
“tricks of the trade” for intermediate to 
advanced users; 


« Provide online forums or a comment function 
for staff and students to submit opinions and 
express their learning and teaching needs 
around the university’s standardized ePortfolio 
system for improvement; 

• Allow for successful growth of simple social 
technologies such as blogs, wikis, rich site 
summary (RSS) feeds, and social networking 
tools, alongside the university’s ePortfolio tool 
to support the use of a variety of digital 
learning experiences that can meet diverse 
learning and technology requirements; 

• Provide examples of rubrics in order to assist 
teachers with the time-consuming nature and 
difficulties around the marking of ePortfolio 
work. 

The results from this project indicate that more can 
be learned from studies such as these, especially around 
the subjects of the impact of ePortfolio-making over a 
longer period of time and of minimizing the digital 
divide for rural-based learning via a wider evaluation 
process. Other related areas of further research might 
include whether or not universities would be wise to 
offer multiple ePortfolio systems for students alongside 
traditional platforms, including hardcopy options if 
broadband is not sufficient. It would also be 
advantageous to conduct a global ePortfolio software 
analysis and collect data from sites such as Electronic 
Portfolio Action and Communication (EPAC) and 
report the findings alongside the internet search data 
collection process used here. Another area of interest is 
to discover what industry’s expectations might be 
regarding students evidencing of learning and graduate 
outcomes for job readiness, especially considering the 
recent Hart Research Associates (2013) report, in which 
“four in five employers say an electronic portfolio 
would be useful to them in ensuring that job applicants 
have the knowledge and skills they need to succeed in 
their company” (p. 3). Finally, we concur with previous 
studies (e.g., Allan & Cleland, 2012) that ePortfolio 
workshops are more beneficial in high functioning 
computer laboratories than in lecture theaters and that 
offering ePortfolio aid wholly online via engaging and 
minimal downloading as well as providing continual 
face-to-face support options at the introductory, 
intermediate, and advanced levels is important for 
sustainable ePortfolio implementation. 
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Appendix 

A Summary of ePortfolio Software Platforms Being Supported in Australia’s Higher Education Sector (2014) 



Name of Institution 

Mahara 

Pebblepad 

Other 

url address as of 2014 

1 

Australian National 
University 

V 

X 

X 

http://cecs.anu.edu.au/files/ePortfolioConferen 

ceHandout.pdf 

2 

Central Queensland 
University*** 

V 

X 

X 

http://www.cqu.edu.au/about-us/leaming-and- 

teaching/office-of-learning-and- 

teaching/resources/learning-technologies/e- 

portfolios 

3 

Charles Darwin 
University** 

X 

X 

Web2.0 

http ://leamline. cdu.edu.au/units/hit3 81 /eportfo 
lio/startup/startup. html 

4 

Charles Sturt University 

X 

V 

X 

http://www.csu.edu.au/division/landt/resource 

s/eportfolio 

5 

Curtin University 

X 

X 

In-house 

https://iportfolio.curtin.edu.au/ 

6 

Deakin University 

X 

X 

Desire2Learn 

Learning Management System log in access 
only 

7 

Edith Cowan University 

X 

V 

X 

http://intranet.ecu.edu.au/leaming/current- 

projects/leaming-portfolio-pebblepad 

8 

Flinders University 

X 

X 

Web2.0 

http://www.flinders.edu.au/eportfolio/ 

9 

Griffith University 

X 

X 

Blackboard 

unknown 

10 

James Cook University 

X 

V 

CareerHub 

http://www- 

public.jcu.edu.au/careers/JCUPRD 034893 

11 

La Trobe University 

X 

V 

X 

http://www.latrobe.edu.au/students/it/teaching/ 

pebblepad 

12 

Macquarie University 

X 

X 

X 

unknown 

13 

Monash University 

V 

X 

X 

http://www.monash.edu.au/news/monashmem 
o/assets/includes/content/20100623/stories- 
more-uni-newsl .html 

14 

Murdoch University 

X 

V 

X 

http://our.murdoch.edu.au/Educational- 

Development/Educational- 

technologies/PebblePad/ 

15 

Queensland University 
of Technology 

X 

X 

In-house 

http://www.studenteportfolio.qut.edu.au/ 

16 

Royal Melbourne 

Institute of 

Technology* 

X 

V 

X 

http://rmit.edu.au/browse;ID=075sqiglpgj0z 

17 

Southern Cross 
University*** 

X 

V 

X 

http://scu.edu.aU/teachinglearning/index.php/7 

9 
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18 

Swinburne University 
of Technology** 

X 

X 

Online 

Galleries 

http://www.swinbume.edu.au/design/portfolio 

s/docs/pdf/GPS_Gradex.pdf 

19 

University of Adelaide 

X 

X 

X 

unknown 

20 

University of Canberra 

X 

X 

X 

unknown 

21 

University of 

Melbourne 

X 

X 

Blackboard 

http://www.lms.unimelb.edu.au/user guides/p 
ortfolio_student_guide .pdf 

22 

The University of New 
England*** 

V 

X 

X 

unknown 

23 

University of New 

South Wales 

V 

X 

X 

unknown 

24 

University of Newcastle 

V 

X 

X 

unknown 

25 

University of 

Queensland 

V 

X 

In-house 

http://www.eleaming.uq.edu.au/content/eporto 

lios-eportfolios-currently-used 

26 

University of South 
Australia 

V 

X 

X 

http://w3.unisa.edu.au/tel/leamonline/eportfoli 

o.asp 

27 

University of Southern 
Queensland*** 

V 

X 

X 

http ://w ww. usq. edu.au/ele/eportfolio 

28 

University of 

Technology Sydney 

X 

X 

In-house 

http://www.iml.uts.edu.au/elearning/eportfolio 

s.html 

29 

The University of 

Sydney 

X 

V 

X 

http://sydney.edu.au/eleaming/staff/getStarted/ 
ePortfolio. shtml 

30 

University of Tasmania 

X 

V 

Desire2Leam 

http ://w ww .teaching- 

learning.utas.edu.au/eleaming/eportfolios 

31 

University of Sunshine 
Coast*** 

X 

V 

X 

http://www.usc.edu.au/university/leaming- 

and-teaching/eportfolios 

32 

University of Western 
Australia 

X 

X 

In-house 

http://www.ecm.uwa.edu.au/ data/assets/pdf 
file/0020/2285201/FASE-Seminar-Flyer-16- 
April-2013.pdf 

33 

University of Western 
Sydney 

X 

X 

X 

unknown 

34 

University of 
Wollongong 

V 

V 

X 

http://staff.uow.edu.au/eteaching/ePortfolio/in 

dex.html 

35 

Victoria University* 

X 

V 

Desire2Leam 

http://leamingandteaching.vu.edu.au/teaching 
practice/blended leaming/eleaming environm 
ent/eportfolio/ 

Total 

10 

12 

14 



Note. V = Used. X = Not used. *Federation University Australia is the only regional multi-sector university and 
**Swinburne University of Technology are two of only five dual-sector institutions in Australia, of which **Charles 
Darwin University is the only one outside Victoria. The others are *Royal Melbourne Institute of Technology and 
*Victoria University. ***There are five rural universities in Australia. 



